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1. Introduction 


Evidence-based curricular programs are the current gold stan- 
dard in early childhood education, linked to improved student 
outcomes and holding potential to reduce the achievement gap 
(Pianta, Barnett, Burchinal, & Thornburg, 2009). In order to effect 
widespread change in student learning, high-quality curricula need 
to be implemented at scale with high fidelity of implementation 
(FOI; Coburn, 2003; Clements, Sarama, Wolfe, & Spitler, 2015). 
Curricular FOI is the degree to which curricula are carried out as 
intended by developers (Berkel, Mauricio, Schoenfelder, & Sandler, 
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2011; Durlak & DuPre, 2008). One critical element of FOI we 
examine in this study is teachers' adherence to the core curricular 
components including materials, routines, and strategies. Unfor- 
tunately, FOI for early childhood curricula often varies by teacher, 
creating appreciably different experiences and ultimately, out- 
comes, for children (Lieber et al., 2009; Zaslow, Tout, Halle, 
Whittaker, & Lavelle, 2010). Thus, supporting FOI is necessary to 
achieve the potential benefits of high-quality curricula. 

Professional development (PD) formats that support curricular 
FOI, such as workshops and coaching (Clements & Sarama, 2008; 
Starkey, Klein, & Wakeley, 2004), require substantial financial and 
human resources (Knight, 2014; TNTP, 2015). In sustainable 
curricular scale-up, districts, as opposed to curricula developers, 
must provide manageable PD supports for FOI that are less resource 
intensive (Coburn, 2003). FOI supports that allow for independent, 
teacher-driven PD engagement (Clements et al., 2015) could sup- 
plement more resource intensive and cost prohibitive PD and in- 
crease feasibility for school districts. 

Internet-based resources allow for the provision of “just in time” 
supports (Romiszowski, 1997), or resources that can be used in the 
moment, while planning to teach a lesson (Dede, Jass Ketelhut, 
Whitehouse, Breit, & McCloskey, 2009; Gill, 2011). Teachers 
perceive pacing, flexibility, and convenient access as benefits of 
online PD (McNamara, 2010). However, provision of more resources 
does not necessarily mean more teacher use, especially in cases 
where teachers have agency over which resources they use and 
how often they use them (Kinzie, Whittaker, McGuire, Lee, & 
Kilday, 2015; Powell, Diamond, & Koehler, 2010). While re- 
searchers are beginning to examine characteristics of coaching that 
support FOI] (Paxton, Wanless, & Rimm-Kauffman, 2013), little is 
known about the relationship between online supports (e.g., 
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demonstration video exemplars) and teachers’ FOI. Isolating and 
examining teachers’ use of online PD resources, their relationship 
with teachers’ FOI, and teachers' perceived usefulness of those 
online supports can inform the design of future online curricular 
support systems. 

In this study, we examine the use of online curricular demon- 
stration videos, video exemplars of high-quality and high-fidelity 
implementation of specific teaching practices (Pianta, Mashburn, 
Downer, Hamre, & Justice, 2008). We seek to explore the rela- 
tionship between early childhood teachers’ use of online demon- 
stration videos and their FOI of the MyTeachingPartner-Math/ 
Science (MTP-M/S) early childhood mathematics and science 
curricula. Observing other teachers’ instruction through videos is a 
common technique for teacher training (Borko, Jacobs, Eiteljorg, & 
Pittman, 2008; Star & Strickland, 2008). However, in studies pub- 
lished to-date, researchers have not yet examined if viewing video 
exemplars is positively associated with teachers' own FOI. 

In order to better understand teachers' use of online PD re- 
sources, we qualitatively examine teachers’ perceptions of the 
usefulness of online demonstration videos in supporting their FOI. 
Using a sequential mixed methods explanatory design, we draw on 
the strengths of numeric data to establish a significant positive 
relationship between demonstration video viewing and FOI. Then, 
we draw on the strengths of narrative data, employing elements of 
grounded theory (Charmaz & Belgrave, 2012; Strauss & Corbin, 
1998), to further understand why teachers choose to use the 
videos or not. 


1.1. Defining and measuring fidelity of implementation 


Berkel et al.'s (2011 ) model of program implementation includes 
fidelity, quality of implementation, adaptation, and participant 
responsiveness. In this study, we focus on curricular fidelity of 
implementation (FOI), or the extent to which teachers adhere to the 
lesson design and carry out the actual components or steps in an 
activity as presented in the curriculum (e.g., Piasta, Justice, 
McGinty, Mashburn, & Slocum, 2015). For example, as illustrated 
in Fig. 1, each MTP-M/S activity includes an “Investigate” section 
comprised of a series of steps to guide students' inquiry (Kinzie 
et al., 2015). One indicator of adherence to an MTP-M/S activity is 
the completion of each investigation step. While we recognize that 
FOI can be considered more broadly as a multifaceted construct 
including but not limited to measures of dosage, quality, adapta- 
tion, and responsiveness (Berkel et al., 2011; Durlak & DuPre, 2008; 
O'Donnell, 2008), we use this narrower, adherence-focused defi- 
nition of FOI based on instructional design of the demonstration 
videos to specifically show activities carried out as intended. Our 
confirmatory questions and hypotheses are based on our belief that 
the videos will have a primary influence on adherence to the 
curricular design. 

FOI is of key importance in interpreting if outcomes are a true 
reflection of an intervention's effect. In others words, they can help, 
“distinguish between intervention failure and implementation 
failure” (Abry, Hulleman, & Rimm-Kaufman, 2015, p. 321). If an 
intervention is delivered with low fidelity, positive effects are not 
necessarily attributable to the intervention, and negative or null 
effects are not necessarily evidence of an unsuccessful intervention 
(Wang, Nojan, Strom, & Walberg, 1984; Durlak & DuPre, 2008). 
Investigations of preschool science (Gropen, Clark-Chiarelli, Ehrlich, 
& Thieu, 2011), literacy (Davidson, Fields, & Yang, 2009), and 
mathematics (Kinzie et al., 2015) curricula have found higher FOI to 
be positively associated with gains in students’ skills. Findings such 
as these lend support for the importance of designing curricular 
support packages to specifically target teachers' FOI and examining 
FOI as a moderator of curricular effects. 


1.2. Fidelity of implementation for sustainable curricular scale up 


Sustainable curricular scale up — the degree to which curricula 
can be implemented over time in diverse contexts with continued 
fidelity to the curricular design (Coburn, 2003; Dede, 2006) — is of 
growing concern in the field of educational research (Century & 
Cassata, 2014). Ultimately, school districts need to sustain 
curricula with fewer financial and human resources than may have 
been available in the initial implementation (Fishman, Marx, 
Blumenfeld, & Soloway, 2004). Lack of resources might contribute 
to low levels of program sustenance, as in the case of First Step, a 
program supporting students' replacement of problem behaviors 
with positive behaviors both at school and home (Sumi et al., 2012). 
Despite positive academic and behavioral outcomes, only one of 
five sites maintained use of the program after a scale-up trial. 

Recent studies and discussions of curricular sustainability 
emphasize the continued importance of FOI (Clements et al., 2015; 
Fishman, Penuel, Hegedus, & Roschelle, 2011; Layzer, 2013) beyond 
initial establishment of positive curricular effects. Sustained FOI is 
moderated by teachers’ initial degree of implementation fidelity 
(Clements et al., 2015; Lieber et al., 2010). Teachers showing high 
FOI at the start of a study are better able to maintain imple- 
mentation fidelity over time (Clements et al., 2015; Wenz-Gross & 
Upshur, 2012) and are more likely to continue to use curricula 
beyond an implementation trial (Lieber et al., 2010). Effective, 
independently accessible, and less resource-intensive supports 
hold promise both for initially increasing and sustaining high 
curricular FOI, maximizing the impacts of evidence-based curricula 
on student outcomes by establishing and maintaining treatment 
strength. 


1.3. Online support for teachers' fidelity of implementation 


Curriculum developers are shifting both formal, in-person PD 
formats (e.g., workshops, coaching), and independently accessible, 
teacher-selected PD resources (e.g., demonstration videos), online 
with promising results (Downer, Kraft-Sayre, & Pianta, 2009; 
Hamre, Pianta, Mashburn, & Downer, 2012; Powell, Diamond, 
Burchinal, & Koehler, 2010). Online PD increases flexibility and 
decreases travel costs while maintaining or extending geographic 
reach (Dede et al., 2009). Although evidence suggests the viability 
of online curricular guidance as an alternative to in-person support 
in terms of the impact on student learning gains (e.g., Powell, 
Diamond, Burchinal et al., 2010), curricular implementation in- 
terventions finding consistently high FOI often provide costly 
formal PD formats such as workshop series and intensive instruc- 
tional coaching (Clements & Sarama, 2008; Knight, 2014; Starkey 
et al., 2004; TNTP, 2015). Online curricular resources, Internet- 
based supports providing instructional guidance and models, are 
a scalable and affordable alternative to promote successful curric- 
ular implementation leading to positive gains in student outcomes 
(Downer et al., 2011; Hamre et al., 2010). 

The provision of online curricular resources also allows for 
increased teacher agency in self-selecting independently accessible 
supports, but there are mixed findings as to their relative value. 
Pianta et al. (2008) found teachers receiving both coaching and 
access to website resources that included video exemplars of high 
quality interactions had significantly greater improvement in 
classroom interactions than teachers with access to the website 
only, controlling for teacher and classroom characteristics. How- 
ever, for those teachers with access to the website alone, higher 
usage rates, measured by page views, were associated with greater 
gains in classroom interaction quality. It seems there was value in 
the website resources independent of coaching, but they were most 
effective when used in a blended model. Contrary to Pianta et al.'s 
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October-Science-W1-A2 


Objectives Topic(s) Use the Lingo 
© Observe and describe how worms © Worm © Soil 
affect their environment Ors * Dig * Compare 


© Tunnel © Contrast 


Materials: ( yy Provided; ™} Not Provided) 


<x Wonderful Worms by Linda Glaser =} Experimental and Control Eco-bottles 


(created in October W1-Al) 
Additional Preparation Required: 


GET READY 


® Check the ecosystems to ensure that changes are visible. 


Teaching Tip: Big idea 

The Eco-bottles are habitats in which the worms live. The worms are dependent upon plants (lettuce) 
for food. They need the soil for protection, moisture, and a place in which to make their tunnels and to 
find food. The worms are dependent on the soil, but they also provide benefits for the soil (e.g., worms 
help water and air flow through soil by digging tunnels; worms break down leaves and grass into things 
that plants can use, etc. 


. Science Chant. 


. Holding up the book Wonderful Worms, -< ask: What do you think this book is 
about? 


“Say: Today we will learn more about how worms affect their environment! We 
will also inspect our Experimental and Control Eco-bottles from our science 
experiment to see whether there are differences in them, and learn how our worms 
have been changing their home! 


. Read Wonderful Worms by Linda Glaser. 


@ Ask students to recall and describe their experience with worms from the previous activity. 


5. < Ask: Do you think our worms are digging tunnels like the worms in the book do? 


© Check the Eco-bottles, comparing and contrasting the progress of bottles with worms to 
bottles without worms. Remind students that we are conducting an experiment because 


when we created the bottles, they were the same except for the worms. ~«What is the 
same about these two bottles? What is different? What do you think could be 
causing the difference? What did the book tell us about...? How do we know it 
is the worms making the changes? 


6. «Say: Today we learned more about how worms offect their environment by 
reading our book on worms and observing changes the worms have been making to 
their home in the bottle! 

What changes did you see in the bottles from last time we looked at them? 
Were they the some? ...different? 

« How did our worms act the same os the worms in the book? 

~ How do worms help the soil ond plants? What do they eot? 
How are the worms like us? How ore they like other animals, such as snakes? 


How are worms different from us? How does the body of the worm help it move 
through the soil? 


Encourage students to pretend they are worms and demonstrate how worms move. 


During outside time, ask questions to get students to think about how worms are helping the 
playground and grounds around the school. 


Dig for worms outside. 


For Students Requiring More Challenge For Students Requiring More Support 


© Encourage students to draw © Spend time one-on-one with students 
observational pictures of the Eco-bottles reading Wonderful Worms and 
showing changes that occurred and comparing the pictures to the Eco 
differences between the two bottles, or bottles. 
to record their observations with Shared 
Writing. 


Fig. 1. MTP-M/S activity plan. All activity plans were provided in print format and available on the website, organized by curricular sequence. 


(2008) findings, Hamre et al. (2012) did not find significantly 
different gains in students' social competence skills between 
teachers who received coaching in addition to website resource 
access and those who receive website resources only. The authors 
did find a similarly positive relationship between time teachers 
spent on a website and students’ social competence skills. While 
the additive value of coaching to website resources is unclear, the 
positive relationship between website usage rates and both teacher 
practices and student outcomes supports the value of online re- 
sources as a promising curricular support. 

In order to reap the identified benefits of online curricular re- 
sources, teachers must engage with them. However, teachers vary 
significantly in their use of provided resources. Downer, Kraft- 
Sayre, et al. (2009) investigated website usage patterns of pre- 
kindergarten (pre-k) teachers engaged in an intervention target- 
ing teacher-child interactions and found that teachers, on average, 
accessed the PD website (lessons plans, lesson materials, and video 
exemplars) for approximately 4—5 h over the school year. But, the 
variance in teachers’ minutes of website use per week was high 
(Year 1 M = 11.2 min, SD = 16.1; Year 2 M = 9.0, SD = 13.1), and 
teachers who received coaching in addition to online resources 
spent significantly more time using the website. There is similar 
range in use when teachers receive explicit direction toward re- 
sources (Hemmeter, Snyder, Kinder, & Artman, 2011; Powell, 
Diamond, & Koehler, 2010). Powell, Diamond, and Koehler (2010) 
found that teachers ranged from 0 to 74 visits to video exemplar 
pages and independently accessed only 20% of recommended video 
exemplars despite links embedded in the coach's written feedback. 


Range in teachers’ use of online curricular supports may be due, 
in part, to demographic characteristics. In a study of pre-k teacher 
coaching and web support PD, Downer, Locasale-Crouch, Hamre, 
and Pianta (2009) identified that older teachers and those with 
higher self-efficacy spent more time on the website resource pages. 
Alternatively, teachers' with more pre-k experience spent less time 
on the website resource pages. 

By supplementing formal, externally driven PD experiences 
with resources intended for independent use, teachers have the 
opportunity for continuous learning focused on their individual 
needs (Jones & Dexter, 2014), an identified component of curricular 
sustainability (Clements et al., 2015). Additionally, the frequency of 
cost-intensive formal PD can be reduced to improve sustainable 
feasibility. However, if resources and attention are to be devoted to 
increasing teachers' positive perceptions and use of online re- 
sources for independent professional learning, their value in terms 
of associated positive changes in teacher practice needs to be 
closely examined. 


1.4. Early childhood educators' need for fidelity of implementation 
support 


Research suggests that early childhood educators need support 
in effectively implementing evidence-based best practices. 
Research conducted in U.S. pre-k settings shows that although most 
programs report having a curriculum, as much as 44% of the 
classroom day is spent in non-instructional activities (Early et al., 
2010; La Paro et al., 2009). Additionally, large scale studies 
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suggest the quality of instructional support is low (Hamre, 2014). 
This may be due, in part, to variation in the efficacy of PD supports 
for teacher's curricular implementation (Lieber et al., 2009). 

Online demonstration videos can be an effective component of a 
PD system (Pianta et al., 2008). They allow for a convenient and 
accessible display of high-quality, authentic practice (Pianta et al., 
2009). Teachers can access them anytime, providing professional 
support when teachers need it (Perry & Talley, 2001). Early child- 
hood teachers take advantage of this logistical affordance, evi- 
denced by the varied timing of their resource access across the day 
and their use of videos throughout an intervention period (Powell, 
Diamond, & Koehler, 2010). However, teachers do not consistently 
take advantage of provided resources at high levels (Downer, Kraft- 
Sayre et al., 2009; Powell & Diamond, 2013). 

Despite varied levels of use, early childhood teachers’ percep- 
tions of online supports are generally positive (Powell, Diamond, & 
Koehler, 2010; Whitaker, Kinzie, Kraft-Sayre, Mashburn, & Pianta, 
2007). Teachers in Head Start classrooms, a nationally-funded 
program to promote the school readiness of children from low- 
income families, engaged in Powell, Diamond, and Koehler's 
(2010) coaching for vocabulary and phonemic awareness instruc- 
tion consistently rated video exemplars high in terms of video 
usefulness. Additionally, early childhood teachers with access to 
website resources that included demonstration videos and teach- 
ing techniques perceived greater utility of an intervention website 
than teachers only receiving access to digital versions of curricular 
plans (Whitaker et al., 2007). As the ultimate goal of curricular 
support is to improve student outcomes through changes in 
teachers' classroom practices, a more nuanced understanding of 
why teachers’ feel curricular demonstration videos are or are not 
useful could inform the design of future video resources to maxi- 
mize teacher use. 


1.5. MyTeachingPartner—Math/Science 


Mathematics and science receive little instructional time in U.S. 
pre-k classrooms (Early et al., 2010; La Paro et al., 2009) and are 
areas in which teachers receive less training and PD support (Parks 
& Wager, 2015). However, early mathematics and science skills are 
foundational not only for students' future mathematics and science 
learning but also their learning in other domains (Grissmer, Grimm, 
Aiyer, Murrah, & Steele, 2010). We designed the MTP-M/S curricula 
and professional support system to improve children's early 
mathematics and science skills by increasing the quality of in- 
teractions and instruction that support children's learning in these 
areas (Kinzie et al., 2014; Whittaker, Kinzie, Williford, & DeCoster, 
2016). 

The MTP-M/S curricula include 132 activities implemented 
across a school year (Kinzie et al., 2015). We designed the activities 
to foster student inquiry and provide contextualized learning of 
mathematics and science concepts, drawing connections between 
the students' world and targeted concepts. The curricula include 
“within activity” supports such as learning and development 
theory-based best practice teaching tips for each activity and ad- 
aptations for differentiated learning. (See Kinzie et al, 2015. for a 
complete description of the curricula development process and 
alignment with preschool mathematics and science research-based 
best practice.) Within-activity supports are accompanied by a 
blended formal, in-person and independent, asynchronous PD 
support system featuring video demonstrations. 


1.5.1. MTP-M/S teacher support system 

The MTP-M/S teacher support system consists of five in-person 
PD workshops across the school year and access to a dedicated 
website. We designed the workshops to increase teachers’ content 


knowledge about children's mathematics and science learning and 
help teachers to plan for and reflect on activity implementation. 
The support system aligns with research-based best practice PD 
recommendations (Desimone & Garet, 2015) suggesting support 
should be sustained throughout the year (workshop spread and 
ongoing Internet resource access), focused on content (children's 
mathematics and science learning), and include active learning 
opportunities (teacher reflection). Further, the focus on mathe- 
matics and science is coherent with a national U.S. effort to improve 
students' mathematics and science skills (National Science and 
Technology Council Committee on STEM Education, 2013). 

The MTP-M/S website includes: (1) video demonstrations of all 
curricular activities, (2) digital versions of activity plans, and (3) 
year-long activity trajectory displays by subject, identifying the 
domain of each mathematics and science activity and illustrating 
the flow of activities by domain across the year. When a teacher 
navigated to an activity, he or she first landed on the associated 
demonstration video page, as shown in Fig. 2. A teaching tip, 
identical to one on the activity plan, accompanied the demon- 
stration video. The complete instructions for facilitating the activity 
were available on a second tab (“Activity Details”). 

Demonstration videos are the only resource unique to the web- 
site, as teachers also received activity plans and trajectories in print 
format. Demonstration videos range in length from 1 min, 29 s to 
6 min, 7 s (M = 3 min, 55 s) and feature high fidelity implementa- 
tions of all activity components that incorporate high-quality 
teacher-child interactions. We intended MTP-M/S activities to be 
10—20 min long, so the demonstration videos show or describe key 
moments in the activity rather than a complete, unedited imple- 
mentation. As teachers and students engage in each of the key 
phases of the activity (Engage, Investigate, and Discuss), text overlay 
identifies the phase and specific step in the activity plan, as shown in 
Fig. 1 with “Investigate: check the Eco bottles.” We used video from the 
pilot implementation of MTP-M/S to create the demonstration 
videos, showing actual teachers implementing the curricula. 

Desimone and Garet (2015) identify the use of catalogued video 
resources used in real-time as teachers plan their lessons as “an 
area ripe for further research” (2015, p. 258). In this study, we are 
specifically interested in the time teachers spent viewing the online 
demonstration videos and if this is related to their FOI. Further, we 
want to understand teachers' perceptions of the demonstration 
videos in terms of their usefulness for curricular implementation 
guidance. 


2. Research questions 


We used both quantitative and qualitative methods to investi- 
gate three research questions: 


1. How often did teachers use the online curricular demonstration 
videos? 

2. What was the relationship of the amount of time teachers spent 
viewing online curricular demonstration videos of MTP-M/S 
activities with their fidelity of curricular implementation? 

3. Within a sample of teachers purposefully selected to represent a 
cross-section of high and low demonstration video use and high 
and low fidelity of implementation, what were their perceptions 
of the usefulness of the MTP-M/S demonstration videos for 
curricular implementation guidance? 


We expected teachers to range in their use of the demonstration 
videos but did not have a specific hypothesis as to the amount of 
time they would spend watching videos. Based on prior work 
examining the relationship of the use of online teacher PD supports 
with growth in teachers’ classroom practices (Pianta et al., 2008), 
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my TeachingPartner 


Math 22: Science 


> 


Worms II 


ACTIVITY DEMONSTRATION 


Investigate: check the Eco-bottles_ 


~ eae =a 


SCIENCI 


Fig. 2. MTP-M/S Demonstration video webpage for Worms II activity. Text overlay on the video highlights the “Investigate” step. 


we hypothesized that time spent watching demonstration videos 
would be positively related to teachers' FOI. Finally, because early 
childhood teachers generally report positive perceptions of online 
curricular supports (Powell, Diamond, & Koehler, 2010; Whitaker 
et al., 2007), we hypothesized that the demonstration videos 
would be viewed as providing useful implementation guidance. 
However, we did not have any predictions about which specific 
components of the demonstration videos would be most useful. 


3. Methodology and methods 
3.1. MyTeachingPartner—Math/Science 


The current study uses data from the first implementation year 
of the two-year randomized, controlled efficacy trial of MTP-M/S 
conducted using two cohorts at two sites. Site 1 classrooms 
(n = 130), participating between 2013 and 2015, were from a va- 
riety of public, private, and Head Start early childhood centers in a 
Midwestern metropolitan area. Site 2 classrooms (n = 40), 
participating between 2014 and 2016, were public pre-k classrooms 
in a southeastern mid-sized city school district. We recruited 
teachers by mailing flyers to preschool program directors and 
holding meetings with teachers, who needed computers to be 
eligible, to inform them about the study and ask for their consent to 
participate. For consenting teachers, we used block randomization, 
equally distributing classrooms across program types and locations, 
to assign teachers to the MTP-M/S treatment condition (n = 87) or 
Business-as-Usual condition (n = 79). 


3.2. Present study sample 


Focusing on support for teachers’ curriculum implementation, 
we only included the 87 teachers assigned to the MTP-M/S treat- 
ment condition in this study's sample, as those teachers had access 
to both the MTP-M/S curricula and associated curricular supports. 
We further limited our sample to teachers who had (1) interacted 
with at least one of the online demonstration videos, and (2) had 


FOI data (see further information below in methods section). This 
led to a final sample of 34 teachers (Site 1: n = 24, Site 2: n = 10) 
(See Table 1). 

We tested for demographic differences between included and 
excluded treatment teachers using independent samples t-tests 
(years of pre-k experience, age, years of education, technology 
habits) and a chi-square test of independence (ethnicity). There 
were no significant differences across demographic variables with 
the exception of teacher age. Excluded teachers were significantly 
younger than included teachers (t [76] = 2.11, p = 0.04). 

At the end of their first study year, we used critical case pur- 
poseful sampling, sampling a small number of high-impact infor- 
mant cases (Patton, 1990), to select four Site 2 teachers for follow- 
up interviews based on preliminary quantitative data analysis. Site 
1 teachers could not be interviewed due to the time elapsed since 
their study participation. To purposefully select interviewed 
teachers, we used a scatterplot to display the correlation between 
the number of minutes teachers spent viewing demonstration 
videos and teachers' FOI. We selected four teachers based on their 
locations within the four quadrants of the scatterplot: Teacher A, 
high video viewing and high FOI; Teacher B, high video viewing and 
low FOI; Teacher C, low video viewing and high FOI; and Teacher D, 
low video viewing and low FOI (See below for measures and 
analysis). We placed the quadrant axes at 0.00 for standardized 
demonstration video viewing minutes and fidelity score such that 


Table 1 
Descriptive demographic statistics for included sample (N = 34). 

Variables M SD Range* 

Age 41.00 11.26 22.00—61.00 
Years of Experience in Pre-k 8.75 8.47 0.00—30.00 
Years of Education 15.09 1.83 12.00—18.00 

Ethnicity” Freq. % 

Black 5 14.7 
White 24 70.6 
Other 4 11.8 


* Range reflects range of reported values. 
> 1 participant in the sample did not report ethnicity. 
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high video viewing and FOI are standardized scores above 0.00. 
Table 2 displays the selected teachers demographic baseline char- 
acteristics and outcome measures as compared to the means for the 
teachers in their respective quadrants. Teachers received fifty dol- 
lars for completing the interview. 


3.3. Data collection procedures 


Data for this study come from fall and spring teacher surveys, 
activity FOI codes based on videotapes, an automated weblog server, 
and semi-structured interviews. We mailed all teachers a paper copy 
of the fall survey, and they returned completed surveys in a pre- 
stamped envelope. We asked teachers to film their implementa- 
tion of all MTP-M/S activities and submit these monthly by mail. To 
obtain an adequate sample of teachers’ activity implementation 
practices across the year (September, October, November, February, 
March, April), across domains (mathematics vs. science), and across 
activity settings (whole vs. small group), we randomly selected two 
tapes for each teacher per month for coding. We coded an average of 
8.24 tapes per teacher (SD = 3.81, Mode = 12), with a range of 2—12 
tapes (the maximum possible). Teachers with less than 12 tapes 
either failed to submit some tapes or submitted tapes that were not 
codable. In order to have FOI outcome data, teachers needed to 
submit at least one codable video of their teaching practice. A 
codable video was at least 8 min, did not clearly cut off the activity, 
was loud enough to hear teacher and student utterances, and never 
had the teacher off camera for more than 60 s. Coding by trained 
research assistants, calibrating weekly with double-coded master 
videos to prevent coder drift, focused on documentation of imple- 
mentation fidelity. We double coded 40% of the tapes, showing 
moderately strong inter-rater reliability (k = 0.78; McHugh, 2012). 
We automatically collected teachers' viewing of demonstration 
videos in a weblog server. The first author conducted and recorded 
semi-structured interviews at the teachers’ schools, asking about 
their use and perceptions of the demonstration videos. Two authors 
(E.A.B. and E.F.) coded all interviews. 


3.4. Measures 
3.4.1. Teacher and classroom characteristics 
The fall teacher survey collected demographic information 


including teachers’ age, ethnicity, years of experience teaching pre- 


Table 2 


k, and years of education. Teachers also completed a technology 
habits scale consisting of three items asking teachers how 
frequently (from 1 “never” to 5 “almost daily”) they “used technology 
to teach in their classroom,” “provided students opportunities to learn 
with technology,” and “used the Internet to assist in their lesson 
planning.” We averaged the three items' scores for a possible score 
between 1.0 and 5.0. Teachers had a moderately high mean score of 
3.83 (SD = 1.02, Range = 1.67—5.00). Parents completed a survey 
collecting information on annual household income, aggregated to 
a classroom mean. For included teachers’ classrooms, the average 
mean classroom annual income was $51,149.33 (SD = $25,356.13). 


3.4.2. Fidelity of implementation 

The MTP-M/S Fidelity Measure primarily assesses teachers' 
adherence to the MTP-M/S curricular design (Kinzie, Whittaker, 
Kilday, & Williford, 2012). Ten items measure teachers' adherence 
to the lesson plan, such as “All main components in the activity are 
completed” (Numbered items in the activity plan, See Fig. 1). The ten 
adherence items evaluated adherence to group size (whole vs. 
small), materials use, warm-up chant use, the engage prompt, the 
main (numbered items) and supporting components (bulleted 
items), teacher and student use of mathematic and scientific lan- 
guage, use of discuss questions, children's engagement in tasks. 
Four items assess teachers’ quality of activity implementation, such 
as “Teacher extends/elaborates on students' comments or questions.” 
The four quality items evaluated the accuracy of teacher mathe- 
matic or scientific explanations, the extent to which teacher's eli- 
cited children's observations and explanations, the extent to which 
children shared those observations and explanations, and the 
extent to which teachers extended on student comments and 
questions. Coders watched each video twice, primarily focusing 
first on adherence items and second on quality items. In this study, 
we found the measure to have acceptable internal consistency 
(a = 0.70; Nunnally & Bernstein, 1994). The measure included four 
dichotomous items and ten 4-point Likert scale items. For all ana- 
lyses, we standardized scores at the item level and averaged, 
equally weighting each item regardless of item level scaling 
(M = 0.22, SD = 4.58, Range = —10.01—7.17). 


3.4.3. Online demonstration video use 
We tracked teachers' use of web-based resources through a 
server that automatically recorded specific information about the 


Demographics and scores for site 2 selected teachers and interview selection quadrant means. 


Selected Teacher Demo Viewing and FOI Age M 


Quadrant (SD) (SD) (SD) 

Teacher A 54 8 40 
High Demo Viewing & High FOI 45.5 7.1 (9.7) 44 (0.5 
n=8) (10.6) 

Teacher B 53 15 5.0 
High Demo Viewing & Low FOI 50.7 13.3 (10.1) 44 (0.5 
n= 3) (7.8) 

Teacher C 35 13 43 
Low Demo Viewing & High FOI 35.9 8.6 (7.2) 3.5 (1.0 
n=11) (6.4) 

Teacher D 29 1 5.0 
Low Demo Viewing & Low FOI 39.8 8.1 (9.7) 3.6 (1.2 
n=12) (17.7) 


* Fidelity scores are standardized. 


Years of Pre-K Exp. M Tech Habits M Demo Viewing Minutes 


FOI Score* M_ Spring Survey Demo Video 


M (SD) (SD) Perceptions” 
Add to Worth Easy to 
Practice Time Use 
254.30 Te 5 5 5 
186.4 (127.0) 4.6 (1.7) 4.7 (0.5) 4.7 (0.5) 49 (0.5 
110.38 27 5 5 5 
83.18 (25.6) —5.7 (3.8) 4.0 (1.4) 40(14) 40(14 
0.75 3.7 3 3 3 
7.20 (15.1) 3.0 (1.8) 3.7 (0.8) 3.6(1.1) 4.1 (0.7 
0.73 —44 - = = 
10.21 (9.5) —3.7 (2.8) 3.8 (1.6) 3.5(1.5) 44(0.9 


> Not all teachers completed the spring survey such that each quadrant has a different n for these means: High Demo Viewing & High FOI n = 7; High Demo Viewing & Low 


FOI n = 2; Low Demo Viewing & High FOI n = 10; Low Demo Viewing & Low FOI n = 8. 


© The selected teacher for Low Demo Viewing & Low FOI did not complete the spring survey. 
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length of time that teachers spent watching demonstration videos. 
Server data included the video actions of play, pause, seek (forward 
and backward), end video, leave page, and close window, all with 
associated time stamps. These allowed us to calculate the number 
of minutes each teacher spent watching demonstration videos 
across the year, removing any time the video was paused and ac- 
counting for seek actions (removing when seeking forward and 
adding when seeking backward and rewatching). If a teacher 
watched a video more than once, all minutes spent viewing were 
included in his or her total time, as we felt the teacher was meeting 
his or her self-perceived need by watching the video again. 


3.4.4. Perceptions of the demonstration video usefulness 

We used a semi-structured interview protocol to ascertain 
teachers’ perceptions of the demonstration videos' usefulness. The 
protocol began with a broad question about teachers' planning 
practices, “Can you tell me about how you prepared to teach MTP-M/S 
activities?” Then, we explained our interest in learning more about 
their use of online curricular supports, specifically the demon- 
stration videos. Questions targeted teachers’ patterns of use for 
videos (e.g., If you watched demonstration videos, when did you 
typically watch them? Why did you decide to watch a demonstration 
video?) and their perceived value of videos (e.g., Can you tell me 
about what you expected to learn from the demonstration videos? 
Were demonstration videos useful to your practice? If so, how or which 
part?). We also used three quantitative items from the spring sur- 
vey to triangulate teachers’ general perceptions of the demon- 
stration videos: “The MTP-M/S Demonstration Videos (1) added value 
to my practice, (2) were worth the time I spent on them, and (3) were 
easy to use.” Possible responses ranged from 1, “Strongly disagree,” 
to 5, “Strongly agree.” 


3.5. Analyses 


To address research question one, we examined teachers’ use of 
the online demonstration videos in terms of minutes spent viewing 
demonstration videos. To address research question two, we 
investigated the relationship of teachers' use of the demonstration 
videos with their FOI using multiple regression with teacher as the 
unit of analysis. Covariates in the model included site, classroom 
mean family income, and teacher characteristics (years of experi- 
ence teaching, years of education, age, ethnicity, and technology 
habits). 

Two teachers did not have data for at least one covariate, so we 
used full information maximum likelihood estimation in Mplus 
version 7.0 to account for missing data. We included teachers’ age 
and technology habits in the model based on previous findings 
from MTP-M/S data that both teachers’ age and technology habits 
(e.g., frequency of using the Internet in planning) were associated 
with their use of website supports (Furnari, Whittaker, Kinzie, & 
Barton, Unpublished manuscript). We also included teachers’ 
years of experience based on Downer, Locasale-Crouch et al.'s 
(2009) findings that teachers with more years of experience spent 
less time and older teachers spent more time using website 
resources. 

To address research question three, we used elements of 
grounded theory (Charmaz & Belgrave, 2012; Strauss & Corbin, 
1998) to analyze the interviews, moving from open coding to 
axial coding (Strauss, 1987). While we drew from grounded theory 
analytic techniques, we do not present a full formed theory and 
believe further data collection is necessary to do so. The first author 
read through all interview transcripts to identify any emergent 
themes. Initial open coding did not restrict themes to those related 
to demonstration videos in order to allow all patterns to surface, 
inclusive of those that may at first seem irrelevant. We reorganized 


the initial set of themes into categories related to video demon- 
stration use and perceived usefulness, removing several sub- 
categories we determined to be unrelated to the use of 
demonstration videos (e.g. descriptions of other classroom 
curricula). The first author then reread the interviews, applying the 
refined categories and subcategories through axial coding. Another 
author on the paper applied the same axial categories and sub- 
categories to establish reliability of code application and identify 
any additional themes missed in the first coding cycles. Finally, we 
met to discuss any discrepant code applications and came to full 
consensus (Forman & Damschroder, 2008) on both the categorical 
codes and the application of those codes. 


4. Results 
4.1. Teachers' use of demonstration videos 


Descriptive analysis of the online server data revealed that 
teachers watched demonstration videos during their first year of 
implementation for approximately 1 h across the school year 
(M = 5714 min, SD = 9646 min), with a range of 
0.0 min—445.88 min. The distribution of demonstration video 
viewing minutes was positively skewed, such that the majority of 
teachers fell between 0.0 min and 153.6 min (2.6 h), one standard 
deviation above the mean. The teacher with 0.0 viewing minutes 
interacted with the demonstration video player (e.g., used seek to 
move through a video) but never actually played a video. The 
teacher with 445.88 viewing minutes watched almost 7.5 h of 
videos across the school year, 4.03 standard deviations above the 
mean. Fig. 3 illustrates teachers’ viewing minutes by month across 
the school year. 


4.2. Relationship between fidelity of implementation and 
demonstration video viewing 


A multiple regression analysis identified a significant positive 
relationship between the minutes a teacher spent viewing 
demonstration videos and his or her mean fidelity score, after 
controlling for all covariates (See Table 3). For every one standard 
deviation increase in demonstration video viewing minutes 
(96.46 min), teachers FOI scores increased by 0.64 standard de- 
viations (2.93 points). This is approximately one and a half hours of 
additional professional learning time for more than a half standard 
deviation increase in implementation adherence. With an 
approximate average of 4 min per video, this is analogous to 
watching approximately 24 demonstration videos out of the library 
of 132 videos. The R? for the overall model was 0.43, indicating that 
the entire collection of predictors was able to explain 43% of the 
variance in the mean fidelity scores. 

Two of the covariates were also significant predictors of teach- 
er's average fidelity: teachers' age and their technology habits, 
controlling for the other covariates and demonstration video 
viewing minutes. For every one standard deviation increase in 
teachers' age (11.26 years), their average FOI decreased by 0.52 
standard deviations (2.38 points). For every one standard deviation 
increase in teachers’ technology habits score (1.02), their average 
FOI decreased by 0.33 standard deviations (1.51 points). 


4.3. Perceptions of demonstration videos for activity 
implementation 


We identified three categorical themes in teachers' responses to 
interview questions about their use of demonstration videos to 
support activity implementation: (1) video use drivers, (2) video 
use deterrents, and (3) video usefulness characteristics. The first 
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Fig. 3. Teachers’ demonstration video viewing across the school year. The bar graph illustrates the sum of demonstration video viewing minutes for all teachers by month. Both 
August and June are included because Site 1 schools begin in August and Site 2 schools finish in June. 


two categories, video use drivers and video use deterrents, capture 
why teachers voluntarily accessed and viewed demonstration 
videos. The third category, video usefulness characteristics, cap- 
tures teachers' perceptions of how the videos they chose to watch 
supported their implementation. 


4.3.1. Video use drivers and deterrents 

Teachers seemed to be driven to select and use demonstrations 
videos for two reasons: targeted video selection and routine prepa- 
ration (Table 4). Three of the four teachers, all except Teacher D (low 
video viewing, low FOI), described targeted video selection, choosing 
to watch a specific video based on previewing the written activity 
plan. Sometimes, this was motivated because teachers found a di- 
rection confusing and sought clarity for the printed activity plan 
(See Fig. 1). For example, Teacher C (low video viewing, high FO!) 
describes, “... There was a couple that I had to go back and reread ... 
if I wasn't clear, then I would go to a video for support.” Her 
description suggests that in order to teach particular lessons with 


Table 3 
Standardized multiple regression predicting teachers’ mean fidelity score. 
8 SE B Pp 
Demonstration Viewing Minutes 0.64 0.15 <0.001 
Covariates 
Site —0.09 0.28 0.749 
Teacher Years of Experience 0.23 0.17 0.169 
Teacher Years of Education 0.19 0.19 0.301 
Teacher Age —0.52 0.18 0.004 
Teacher Ethnicity* 
Black —0.22 0.17 0.199 
Other —0.18 0.14 0.189 
Teacher Technology Habits —0.33 0.13 0.013 
Mean Classroom Annual Income —0.41 0.24 0.083 
R? (SE R?) 0.43 (0.13) 0.001 


Note. Bold text indicates statistical significance of the coefficient. 
* Coefficients are categories of Black and Other tested against White as the 
reference category. 


high fidelity she needed the additional video support, but that was 
not the case for all activities, likely reducing her total video viewing 
time. Targeted video selection was also driven by teacher lack of 
familiarity with activity content. Teacher B (high video viewing, low 
FOI) describes how she chose to watch a video of a lesson on 
counting with a ten-frame, “I hadn't taught counting before. So, 
that was all new, and | just didn't understand that.” She further 
explained that by watching the video, she learned an instructional 
strategy for a content objective, counting, she had not yet taught 
and continued to use that strategy throughout the curriculum. 

The second motivation for video selection was routine prepara- 
tion, systematic viewing of the demonstration videos as part of the 
planning process. Only one teacher, Teacher A (high video viewing, 
high FOI), described this viewing pattern, believing she watched 
almost every video for the lessons she taught in her first year of 
teaching the MTP-M/S curriculum. Teacher A also described 
repeated viewings and sharing the videos with her teaching as- 
sistant. Combined, all of the demonstration videos total approxi- 
mately 515 min (about 8.5 h), and Teacher A watched 
approximately 7.5 h of video across the school year. 

Interviewed teachers described two primary video use de- 
terrents: a lack of time and difficulty relating to depicted classrooms 
(Table 4). Only Teacher D (low video viewing, low FOI) described a 
lack of time as the reason she did not choose to watch videos, 
despite seeing potential value in the videos. “I'd like more time in 
my day, because | think they probably would help me at least un- 
derstand ... I think watching the video I would probably become 
more comfortable with the lesson itself.” Teacher C and Teacher D, 
both teachers with low video viewing, described the second 
deterrent, difficulty relating to depicted classrooms. Teacher D (low 
video viewing, low FOI) primarily described the number of students 
as unrealistic. “It's hard because I have 18 kids where it looks like 
half of them only have like 12 kids in their class, and that makes a 
huge difference,” and Teacher C (low video viewing, high FOI) 
described the demographics of students in the videos as unlike her 
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“If I felt that there was a lesson that I needed more support on, then I would watch the videos.” — LV, HF 


“Usually twice, sometimes three times, especially if | need my assistant to see what I saw, because my assistant might not be in 


“I'd like more time in my day, because I think they probably would help me at least understand ... I think watching the video I 


would probably become more comfortable with the lesson itself.” — LV, LF 


“... the population of children is a little different. So, it's kind hard to determine ... how much they'll understand and how they'll 


Table 4 
Interviewed teachers’ perceptions of demonstration video drivers and deterrents. 
Video Drivers Teachers Examples 
Mentioning 
1. Targeted Video Selection HV, HF 
HV, LF 
LV, HF 
2. Routine Preparation HV, HF 
the planning stage of the lesson.” — HV, HF 
Video Deterrents Teachers Examples 
Mentioning 
1. Lack of Time LV, LF 
2. Difficulty Relating to LV, HF 
Depicted Classrooms LV, LF react to a lesson.” — LV, HF 


“ 


own, “... our population [of students] is similar among teachers 
throughout Briarwood.! So, I think it would be better to watch ... 
maybe videos of our colleagues teaching the lesson.” However, 
Teacher B (high video viewing, low FOI) noted the difference in the 
student behavior featured on the videos but did not seem to be 
deterred by this, suggesting some teachers may still relate to videos 
despite situational differences. 


4.3.2. Video usefulness 

We identified eight themes as characteristics of video usefulness 
from interviews with Teacher A, Teacher B, and Teacher C (See 
Table 5). Teacher D (low video viewing, low FOI) only spoke hy- 
pothetically about what she thought might be useful, as she could 
not remember independently watching demonstration videos. The 
emphasis on positive video aspects aligns with the generally pos- 
itive perception expressed by teachers on the spring survey items 
(Table 3). Teachers A and B with high video viewing rated the 
videos as 5 for their impact on teaching practice, being worth the 
time to watch them, and ease of use. Teacher C, low video viewing 
and high FOI, rated the videos as 3 for all items, logically less than 
the high video users. Teacher D, low video viewing and low FOI, did 
not complete the survey. 

The three most frequently mentioned usefulness themes were 
enactment of written activity plans, adaptation of the activity, and 
expected student behavior. Enactment of written activity plans en- 
compasses references to the visual or aural representation of the 
activity as being beneficial to supplement the written description. 
Teacher A and Teacher B, both with high video viewing, explicitly 
described themselves as being visual learners,” seeking an image to 
accompany the text prompts and directions. Adaptation of the ac- 
tivity refers to instances when a teacher watched a video and made 
a purposeful choice to change the activity based on what she saw. 
This included changing the group size format (small vs. whole), 
modifying the materials or arrangement of materials, rearranging 
the order of activities, and adjusting the content based on student 
abilities. Interviewed teachers described making preemptive 
changes to the activity based on their physical classroom layout, 
specific students, or group behavior dynamics (e.g., difficulty sitting 
for discussion). Expected student behavior refers to instances when 
teachers were unsure of how their students would react to an 
aspect of a lesson, such as live worms in a science activity, or spe- 
cific questions. Interviewed teachers described anticipating their 


1 District name changed to maintain anonymity. 

? Although recent literature (Rohrer & Pashler, 2012) has not found sufficient 
evidence to support the notion that individuals learn best through one channel, this 
is the perception of the teachers. 


own students’ possible reactions based on those of the students in 
the video. 

The next two most frequently mentioned usefulness themes 
were organization and use of materials and benchmark expectation of 
implementation. Interviewed teachers described how they were 
unsure as to how the materials should be prepared or arranged for 
the lesson. By watching the video, they were able to see one or 
more examples (many videos feature multiple teachers) that were 
high fidelity materials arrangements. Teacher B (high video 
viewing, low FOI) described how she did not know how a 10-frame 
was used for counting, “so then when I watched her, then of course 
it made sense the rest of the year.” Teacher A and Teacher B, both 
high video viewers, sought information about what was expected 
of them by the researchers in implementing the lesson. They saw 
the demonstration videos as a benchmark of what researchers were 
looking for in lesson implementation. 

The final three usefulness themes, highlighting challenging fea- 
tures, questioning model, and timing and pacing of the activities, were 
each only mentioned by one teacher. Teacher A (high video 
viewing, high FOI) appreciated when the videos included a voice- 
over highlighting an important procedural or conceptual feature of 
the lesson, such as the necessity of counting one object at a time. 
Teacher C (low video viewing, high FOI) repeatedly discussed her 
discomfort with the questioning aspect of the activities and sought 
information about the question-driven discussion with students 
and their reactions. Timing and pacing of the activities is an example 
of how videos could be adjusted in the future to be more useful. 
Teacher A (high video viewing, high FOI) repeatedly mentioned a 
desire to get a sense of timing and pacing from the videos, an 
expectation she had for the videos that was not met due to their 
brevity. 


5. Discussion 


Fidelity of implementation is critical for establishing and 
maintaining efficacy of a curriculum (Durlak & DuPre, 2008), but 
there is little research on specific strategies to support FOI. Seeking 
to fill this gap, we examined the relationship between the time 
teachers spent viewing demonstration videos and their FOI. Con- 
firming our hypothesis that time spent watching demonstration 
videos would be positively related to teachers’ FOI, we found a 
strong positive relationship between the two. Although this is not a 
causal relationship, the possibility that viewing demonstration 
videos, independently accessible and low-cost sustainable re- 
sources, could increase teacher FOI is promising given the need to 
sustainably move curricula to scale and maintain high-fidelity 
implementation over time (Clements et al., 2015; Coburn, 2003). 
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“Seeing it, being a visual learner, I think. Seeing it and when you're teaching it, thinking, ‘Oh yea, she did this.”” — HV, LF 
“... how long I think it's going to take me to do that lesson and I try to look at a way to incorporate some of those questions at the 
“... to maybe get a better understanding of how to teach the lesson and how to ask questions and to see actually how the kids 


“I did the one where it's the number matching, and I'm like, ‘How did they line the numbers up? Was it important to have the 
numbers go from 0 to 6 or is it okay to put 0 to 4 and then drop it down?’ That sort of thing.” — HV, HF 


“I think it's easier ... cause it was so new last year, just watching somebody else do it and ... seeing what's expected or what 


“... from time to time when ... somebody speaks [on a voiceover] as to what needs to be done because maybe the person didn't 
bring up that... part of the lesson. I think that is very valuable ... it was something like, umm, ‘Make sure you are counting the 


orange squares and indicate how important it is to start at the bottom and go up’ —HV, HF 


Table 5 
Interviewed teachers' perceptions of demonstration video usefulness characteristics. 
Video usefulness themes Teachers Examples 
Mentioning 
1. Enactment of Written HV, HF 
Activity HV, LF 
LV, HF 
2. Adaptation of Activity HV, HF 
HV, LF end throughout the lesson.” — HV, LF 
LV, HF 
3. Expected Student Behavior HV, HF 
HV, LF react to it.” — LV, HF 
LV, HF 
4. Organization and Use of HV, HF 
Materials HV, LF 
5. Benchmark Expectation of | HV, HF 
Implementation HV, LF you're looking for.” — HV, LF 
6. Highlighting Challenging HV, HF 
Features 
7. Questioning Model LV, HF 
little bit.” — LV, HF 
8. Timing and Pacing of the HV, HF* 


Activities 


In order to maximize on the potential that demonstration video 
viewing affords in supporting FOI, teachers need to view the videos. 
Teachers interviewed in this study described two patterns of 
demonstration video use: as a systematic component of their 
planning or as a targeted supplementary support, used when 
necessary. We predicted teachers would range significantly in the 
time they spent watching demonstration videos. Again, confirming 
our hypothesis, data from this study suggest teachers vary signifi- 
cantly in their degree of use; their average viewing time was about 
1 h but ranged substantially from zero to approximately 7.5 h. This 
usage pattern is similar to the inconsistent use of web resources 
found in earlier studies of curricular support websites (Downer, 
Kraft-Sayre et al., 2009; Hemmeter et al., 2011; Powell, Diamond, 
& Koehler, 2010), despite generally positive perceptions of website 
resources (Powell, Diamond, & Koehler, 2010; Whitaker et al., 
2007). The significant range in teachers’ use of demonstration 
videos impacts the potential for widespread effect, a concern when 
planning future support systems. 

The periodic use of demonstration videos throughout the year 
meets one of the key characteristics of high-quality PD, duration 
over time (Desimone & Garet, 2015), by spreading out the hypo- 
thetically equivalent workshop hours over the course of a year. 
However, not all teachers watched video each month. One strategy 
for increasing the consistent use of resources such as demonstra- 
tion videos would be to track and recognize the additional pro- 
fessional learning time teachers invest, as is done with digital 
badging systems (Diamond & Gonzalez, 2014). It seems teachers 
may need more support and guidance in using the online resources 
effectively, and a blended approach that combines fewer formal, 
leader-driven professional learning, with independent, teacher- 
driven, professional learning is likely appropriate. This aligns with 
research recommending a facilitator to scaffold teachers’ video use 
(Arya, Christ, & Chiu, 2014). However, findings that teachers do not 
necessarily use videos chosen specifically for them (Hemmeter 
et al., 2011; Powell et al., 2010) suggest it may be more valuable 
to train teachers by modeling the use and benefits of independently 
accessible resources. This is also a less time and financial resource- 
intensive strategy than individually selected videos for each 
teacher. 


“how to ask questions and to see actually how the kids react to it ... maybe if there's something that needed to be changed up a 
“... we were just, I was looking for all kind of answers to the timing of the lessons.” 


Teacher described as what she would have liked to get from demonstration videos but did not actually get. 


Teachers in this study identified drivers and deterrents to 
demonstration video use, as well as eight characteristics of use- 
fulness for implementation guidance. While we believed teachers 
would generally perceive the demonstration videos positively, 
we did not hold any a priori hypotheses as to the elements of the 
videos they would identify as more or less useful. To increase 
teacher use of demonstration videos, future curricula developers 
might design videos based on identified usefulness characteris- 
tics, highlight these features during training, and support 
teachers’ time allocation to include video viewing into planning. 
User-centered design of curricular support systems allows for 
feedback loops between teachers and developers in order to 
produce resources better aligned to user needs (Kinzie et al., 
2006), suggesting the value of teacher-identified usefulness 
characteristics for the creation of future demonstration video 
exemplars. For example, Teacher C identified models of ques- 
tioning with students as a feature of the videos, but she was 
actually a low video user. Extended questioning sequences were 
not featured in the demonstration videos to keep them short, but 
providing additional videos to explicitly model high fidelity 
questioning sequences might increase use. The second inter- 
viewed teacher with low video viewing, Teacher D, primarily 
identified time as her barrier to use. The video clips were 
designed to be brief, ranging in length from 1:29 to 6:07 witha 
mean length of 3:55, but teachers may need more support in how 
to structure their planning to incorporate these videos. They 
might be better able to see the value of demonstration videos 
before beginning a new curriculum if the elements of usefulness 
and planning strategies are incorporated into initial training. Of 
note, however, is the restriction of our interview sample to those 
teachers that had interacted with the video player. It is possible 
that complete nonusers would identify additional drivers, de- 
terrents, or usefulness characteristics, an area for future 
investigation. 

While demonstration video viewing had a positive relation- 
ship with teachers' FOI, two of the included covariates had a 
negative relationship with FOI: teachers’ age and technology 
habits, a measure of technology use frequency. This suggests that 
older teachers could use more targeted FOI support. Older 
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teachers recognize the value of professional development for the 
school as a whole more than for themselves specifically (Hustler, 
Mcnamara, Jarvis, Londra, & Campbell, 2003), so highlighting 
personal usefulness of support resources, as described above, 
might prove helpful for these teachers. Internet-based resources, 
including but not limited to demonstration videos, may be an 
avenue by which to reach older teachers, as they engage more 
frequently with curricular websites than their younger counter- 
parts (Downer, Locasale-Crouch et al's., 2009). Similarly, Internet- 
based resources might be an appropriate delivery medium by 
which to provide FOI support to teachers who already frequently 
engage with technology. It is possible that they use the Internet 
to find additional Internet-based resources that may be contra- 
dictory to the curriculum, reducing FOI, so replacing those re- 
sources with developer-provided online resources, well- 
supported in training, could improve FOI. Use of a technology- 
based digital badging system, as described above, might be a 
motivator for teachers already familiar with and using technol- 
ogy. Negative relationships between characteristics of teachers 
and FOI lend further support to the continued investigation of 
demonstration videos as FOI supports, in addition to the inves- 
tigation of other Internet-based support resources. 


5.1. Limitations and future directions 


The current study speaks only to the relationship of demon- 
stration video viewing and FOI during the first year of curriculum 
implementation. However, it is possible that the relationship 
changes as teachers become more familiar with the curriculum in 
subsequent implementation years or when the ownership of the 
curriculum shifts primarily to the school district leaders for sus- 
tainable scale-up. Given the ease of access and low cost of 
demonstration videos once they are created, it is feasible for 
teachers to continue to use them. Further investigation of the effect 
of demonstration video viewing should examine this relationship 
over time, exploring the possibility of a nonlinear relationship. 
Additionally, demonstration videos may serve as a “catch-up” 
strategy for new teachers who were not present at initial trainings, 
rather than simply handing them a written curriculum. 

A limitation of this study is the small sample size of interviewed 
teachers. Therefore, we did not seek to make generalizable state- 
ments, purposefully selecting critical cases based on theoretical 
representation of the quantitative data (Marshall, 1996; Patton, 
1990). Increasing the number of interviewed teachers to confirm 
and expand usefulness characteristics of the videos would increase 
the transferability of findings to a more generalizable sample of 
teachers. Due to the negative relationship between teachers’ age 
and FOI, a purposeful sample of older teachers to identify poten- 
tially different characteristics of usefulness would also help inform 
targeted support for this subset of teachers for whom FOI is 
particularly low. Continuing to use sequential explanatory mixed 
methods in future work examining online FOI supports would build 
a rich body of literature to inform purposeful curriculum design 
decisions. 

The majority of teachers interviewed spoke of the videos as 
informing their adaptation of the lesson. However, the MTP-M/S 
Fidelity Measure is limited in that it does not specifically assess 
whether and how lessons were adapted. Adaption of curricula to 
meet contextual needs is particularly important given the desire to 
scale-up curricula within the context of classrooms and students 
with diverse learning needs (Durlak & DuPre, 2008; Lendrum & 
Humphrey, 2012). Measuring adaptation as part of FOI can help 
to identify the types of adaptations that teachers are making, and 
where adaptation is and is not beneficial for students’ learning 
(Blakely et al., 1987; Dusenbury et al., 2003). 


6. Conclusions 


The positive association between teachers’ online curricular 
demonstration video viewing and their FOI suggests demonstration 
videos are a viable fidelity support resource worthy of further 
investigation. Although evidence-based curricular programs are 
linked to improved student outcomes (Pianta et al., 2009), these 
curricula need to be implemented with fidelity. Identifying types of 
effective curricular resources for teachers is one step toward 
increasing FOI. However, it is not only critical to create those re- 
sources but also to encourage and support teachers’ use of 
independently-driven PD activities. 
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